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Abstract 

Use of appropriate measurement methods and the INTASC Principles to assess teacher 
dispositions provides a safety net against legal and political challenges while also providing 
useful information for outcomes assessment. This paper is a companion to our other 
presentation which discusses technical construction and reporting components of the DAATS 
assessment system. 



Introduction 

The NCATE Requirements, the INTASC Principles (Standards), and the NEA 
Code of Ethics 



NCATE (2002) requires the measurement of dispositions as part of its accreditation 
requirements for teacher education programs. The first standard, entitled, “Candidate 
Knowledge, Skills, and Dispositions,” requires that: “Candidates preparing to work in 
schools as teachers or other professional school personnel know and demonstrate the content, 
pedagogical, and professional knowledge, skills, and dispositions necessary to help all 
students leam. Assessments indicate that candidates meet professional, state, and 
institutional standards.” 

Fortunately, guidance is provided to the community by the common set of national 
standards developed by the Council of Chief State School Officers and promulgated by the 
Interstate New Teacher Assessment and Support Consortium (INTASC) in the form of ten 
principles. Each of the principles includes indicators written at the knowledge, skill, and 
dispositional levels, forming constructs that colleges are required to measure. 

The supporting explanation for this Standard links NCATE and INTASC together, as 
the standards “of reference” in the following statement, making it clear that the INTASC 
Principles should be used as a major basis for designing assessment systems: 

. . .NCATE and INTASC expect teacher candidates to demonstrate knowledge, 
skills, and dispositions 11 to provide learning opportunities supporting students’ 
intellectual, social, and personal development, (p. 1 8) 

1 1 This list is based on the standards of the Interstate New Teacher Assessment and Support 
Consortium (INTASC). The complete INTASC document includes knowledge, dispositions, 
and performance related to each principle. It is available on the website for the Council of 
Chief State School Officers (CCSSO) www.ccsso.org/intasc.html 
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Regrettably, many institutions fail to consider the INTASC Principles when 
developing their processes for measuring teacher dispositions. Instead, they rely on generic 
traits, often linked to morality and ethics that leave them wide-open for attack in the 
measurement and legal worlds. This failure to attend to standards-based definitions and 
adequate due process procedures, when combined with a lack of solid data to diagnose and 
mediate, leaves institutions wide open for attack if they chose to deny graduation to a teacher 
candidate who exhibits dispositional deficits. This emphasis is due, in large part to a 
statement in the NCATE Supporting Information explaining the standard that refers 
institutions to the NEA Code of Ethics and a definition of dispositions that focuses on 
morality and ethics. The NCATE (2002) definition follows: 

The values, commitments, and professional ethics that influence behaviours toward 
students, families, colleagues, and communities and affect student learning, 
motivation, and development as well as the educator’s own professional growth. 
Dispositions are guided by beliefs and attitudes related to values, such as caring, 
fairness, honesty, responsibility, and social justice. For example, they might include a 
belief that all students can learn, a vision of high and challenging standards, or a 
commitment to a safe and supportive learning environment, (p. 53) 

The NCATE (2002) supporting explanation that is linked to ethics follows: 

Candidates for all professional education roles develop and model professional 
dispositions that are expected of educators. The unit includes as dispositions the ideal 
of fairness and the belief that all students can leam. Based on its mission, the unit may 
determine additional professional dispositions it wants candidates to develop^ The unit 
articulates professional dispositions as part of its conceptual framework(s). The unit 
systematically assesses the development of appropriate professional dispositions by 
candidates. [1] Professional dispositions are not assessed directly; instead the unit 
assesses dispositions based on observable behavior in educational settings. 

[1] Codes of ethics may be helpful in thinking about professional dispositions and are available from a 
number of professional associations, including the National Education Association (NEA). 

The result, then, is confusion. On the one hand, NCATE links the measurement of 
dispositions to the INTASC Principles, but, on the other hand, it references institutions to an 
ethical code. They are not the same. The INTASC Principles provide positive statements 
about the beliefs and attitudes teachers should demonstrate about the skills addressed in the 
Principles. The Code of Ethics speaks to what teachers should NOT do. 

Measuring Dispositions 

We use tests, products and live observations to measure knowledge and skills. 
Appropriate measurement methods for affect (dispositions) include scales, questionnaires and 
interviews, focus groups, observations, and apperception tests, as well as documentation of 
inappropriate behavior when it occurs. We will discuss these in more detail in the literature 
section of this paper, but for now, we note here that there is a long history of effective 
affective measurement beginning with Thurstone and Chave (1929) in “The Measurement of 
Attitude. ” We note further that NCATE states in its Supporting Explanation for Standard 1 
that: “Dispositions are not usually assessed directly; instead they are assessed along with 
other performances in candidates’ work with students, families, and communities.” (p. 19) 
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The INTASC principles, when combined with appropriate measurement methods, 
provide a safety net that can allow college personnel to do what they need to do — make 
decisions that are less likely to be successfully challenged while, at the same time, providing 
data that can be aggregated to improve the outcomes of their programs. The Principles allow 
us to focus on the positive. Proper measurement can lead to information about what students 
are learning and what they are not learning, so that program improvement is possible. The 
NEA Code of Ethics allows us only to locate ethical violations. Dispositions are different 
from knowledge and skills, and both require our serious attention. 

Definitions of the Term Dispositions 

Katz (1993) helps us to see that dispositions are both voluntary and a habit of mind - 
outside our conscious control with the definition provided in Box 1.6. 

A pattern of behaviour that is exhibited frequently and in the absence of coercion and 
constituting a habit of mind under some conscious and voluntary control, and that is 
intentional and oriented to broad goals. 

Our dictionary definition comes from Miriam Webster Online Dictionary 
(www.miriamwebster.com) and adds not only mood but a tendency to act in certain ways to 
our understanding. 

A prevailing tendency, mood, or inclination; temperamental makeup; the tendency of 
something to act in a certain manner under given circumstances. 

The draft revised NCATE definition, under public review as of this writing, is as 
follows: 

The professional behaviors educators are expected to demonstrate in their interactions 
with students, families, colleagues and communities. Such behaviors support student 
learning and development and are consistent with ideas of fairness and the belief that 
all students can learn. Based on their mission, professional education units may 
determine additional professional dispositions they want candidates to develop. 
Institutions assess professional dispositions based on observable behavior in 
educational settings. 

Note that this new definition removes references values, commitments and 
professional ethics and references to honesty, responsibility and social justice. 

Our definition of dispositions (Wilkerson and Lang, in press) is as follows: 

Dispositions are in the teacher education context are the aspects of teacher affect - 
attitudes, values, and beliefs — that influence the application and use of knowledge 
and skills, as defined in accepted standards of teaching, e.g., the INTASC Principles. 
Teachers who have the affects required of good teachers will have reached the 
“valuing” level in the Bloom and Krathwohl taxonomy. The guiding question that 
focuses this definition is: “What does the teacher believe to be important about 
teaching and being a good teacher? How is s/he likely to act?” 
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Literature Review 



Research-Based Measurement of Teacher Dispositions 

The published literature on measuring teacher dispositions is sparse. This is probably 
at least in part because of the general assessment illiteracy that pervades our culture (Stiggins, 
2000, and Popham, 2004). A recent study by Schulte, Edick, Edwards, and Mackiel (2004) 
confirms that little guidance is provided about measuring teacher dispositions. There is 
instead much in the literature that is skill based rather than values-based (Darling-Hammond, 
2000). There are some exceptions, Wasicsko’s (2004) 20-Minute Hiring Assessment, Holt- 
Reynolds’ (1991) biographical and metaphorical assessments, and several discussions of 
portfolios applied to disposition assessment; all are cited in (Schulte, et al., 2004). The 
Schulte team developed and validated a Teacher Disposition Index (TDI), which is similar in 
some ways to one of the four instruments we model here. There are many differences, 
though, and chief among them is the limitation to a single instrument type. 

The literature on affective measurement has in its roots the taxonomy developed by 
Krathwohl, Bloom, & Masia (1956). There are a number of methods available for affective 
measurement, and, as with cognitive assessments, multiple measures help to increase the 
confidence in decisions. Hopkins (1998) provides a useful list and discussion of affective 
measures, which are organized and summarized below with relevant references to the 
affective assessment literature: 

Selected Response Methods: Selected response methods provide self-reported 
information that is based on the selection of a chosen response from a pre-determined 
pair or set of responses for each item. Here the respondent indicates a level of 
agreement, importance, or some other value-laden judgment for specified 
characteristics, e.g., a belief in children’s capacity to learn. Scales are an important 
method for measuring affect in this way (Anderson, 1988a). There are four types of 
scales generally used: Thurstone agreement scales (Thurstone, 1928; 

Anderson, 1988b), Likert scales (Anderson, 1988c), rating scales (Wolf, 1988a), and 
semantic differential scales (Phillips, 1988). 

Constructed Response Methods: Constructed response methods provide self-reported 
information also, but in this case, the response is written without pre-determined 
choices. The appropriate methods include questionnaires (Wolf, 1988b), interviews 
(Miller & Cannell, 1988), and focus groups (Flores & Alonso, 1995). Items in this 
category are easier to create, but scoring becomes more subjective and, therefore, 
more complex. 

Observed Performance: Observation assessment is another excellent source of data 
(Stalling & Mohlman, 1988). Included in this group are direct observations in the 
classroom, behavioural checklists (completed after multiple observations and products 
have been analysed), and event reports. Clearly, time is a factor for the observations 
and checklists. Event reports provide a unique approach, used to record (and 
hopefully remediate) ineffective behaviours. 

Legal Issues 

Failure to use appropriate measurement methods has resulted in at least three major 
challenges to institutions seeking to deny graduation to teacher candidates based on 
dispositional issues. In the case of Scott McConnell vs. LeMoyne University, the New York 
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District Court of Appeals upheld Mr. McConnell's right to continue seeking his degree after 
he was expelled for writing a classroom management plan in which he advocated for the use 
of corporal punishment and against multicultural education. This was such a "hot topic" that 
it yields 55,000 hits in a simple Google search, (e.g., York, 2006). McConnell prevailed on 
due process, since LeMoyne had no formal process to assess dispositions. His values "leaked 
out" during a class-based assignment. With no process in place to measure or advice, 
LeMoyne also has little opportunity to obtain program improvement level information 
regarding the outcomes of its programs. 

At Washington State University, a similar decision was reached for a student, Ed 
Swan, who expressed views contrary to the defined dispositions requirements of the college. 
This case was targeted by an organization called FIRE (Foundation for Individual Rights and 
Freedoms), and was the subject of many articles on their site (e.g., 

http ://w w w . thefire. or g/index . php/article/68 3 2 . html . Unlike LeMoyne, which had no formal 
process for measuring or documenting dispositional deficits, WSU had an assessment form 
that was made up largely of subjective criteria. FIRE noted the extreme subjectivity of the 
form as well as the decisions made about Swann, and WSU Seeventually reworked the form, 
taking Swann back into their program. FIRE has not yet noted that the new form, too, has 
room for subjectivity in the decision-making process. WSU is, however, linked its revised 
form to INTASC Principles, and this seemed to satisfy the critics. Outcomes measures 
related to standards may be difficult, though, since there are Principles listed by number that 
exceed the number of INTASC Principles. For example, the first statement is linked to 
Principles 14 and 15, and the INTASC Principles end at number ten. 

Brooklyn College, part of the City University of New York, also experienced 
problems when one of its faculty, E.C. Johnson spoke out publicly against their process of 
measuring dispositions, which, again, were not standards-based. He was particularly 
concerned with the issue of social justice and its measurement, pointing to 13 institutions 
which focus their conceptual framework on social justice. His application for tenure was 
sabotaged on the grounds of collegiality. He asserted his academic freedom rights, was 
defended by a conservative organization, and finally was permitted to continue his 
questioning of the morality basis of evaluating teacher dispositions. (Smallwood, 2003; 
Foundation for Individual Rights in Education, 2003) 

Karen K. Siegfried withdrew from the University of Alaska because the faculty told 
her she did not have the right kind of attitude. She had bumper stickers, including anti- 
abortion, that caused her to park her car at remote locations. She had disagreed with one 
professor’s contention that video games make children violent, evoking the response: “We 
don’t need that kind of attitude.” She had expressed her views on gun control and affirmative 
action. With a 3.75 GPA and one year in the program, after her professors told her that she 
lacked the “professional disposition” necessary to be a good teacher, she witdrew. They told 
her she was inflexible, not open to new ideas and non -responsive to other cultures. She 
suspected she was going to be suspended. (Wilson, 2005) 

In addition to the New York Times and a large number of conservative smaller 
newspapers and magazines, coverage of these events is noted in the Chronicle of Higher 
Education and U.S. News and World Report (e.g., Leo, 2005; Wasley, 2006). These 
institutions now may also be targets for FIRE. It has been suggested that using the INTASC 
Principles as a vehicle to embed issues such as social justice into a standards-based 
assessment process can solve this problem (Wilkerson, 2006). 
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Method 



The disposition indicators of the INTASC Principles were used as the constitutive 
definition of teacher affect, with a series of six instruments developed to define the construct 
in operational terms. The process used to develop the instruments was the five-step design 
model we call “Dispositions Assessments Aligned with Teacher Standards,” or DAATS. The 
steps of the model are: 

Step 1: Define purpose/use, content, and other contextual factors. 

Step 2: Develop a valid sampling plan. 

Step 3: Create instruments aligned with standards and consistent with the sampling 

plan. 

Step 4: Design and implement system. 

Step 5 : Ensure psychometric integrity 

The implementation of the model was carefully constructed using the Standards of 
Educational and Psychological Testing (AERA, APA, and NCME, 1999). These Standards 
are the foundation of all standardized and other test design. When legal challenges are made 
to testing decisions, it is these Standards that serve as the basis for making decisions about 
the validity, reliability, and fairness of the inferences questioned in the court. Thus, by using 
the Standards to guide the process, the instruments were created to withstand legal 
challenges. The integration of the DAATS design model and the AERA, APA, and NCME 
Standards is documented systematically (Wilkerson and Lang, in press). 

Results 

In its current stage of development, there are six instruments, each of which has been 
field tested. These are discussed in more detail by Lang and Wilkerson (2007) and presented 
in very briefly below: 

Disposition Event Report : 

Description: This form provides a record of the teacher’s demonstration of a 
negative disposition, sometimes correlated with the Code of Ethics, sometimes 
with a lack of valuing of skill-based attitudes. 

Status: This form is used intermittently to document problems and 
remediation efforts. It is not, therefore, subject to traditional field-testing 
procedures. 

Belief Scale : This is a self-report 50-item Thurstone agreement scale., aligned with 

each of the INTASC Principles. 

Description: This is a self-report 50-item Thurstone agreement scale., aligned 
with each of the INTASC Principles. 

Status: This form has been field tested more extensively than any of the other 
forms, at four different institutions with about 2,000 examinees. The results 
have been calibrated using the Rasch model of Item Response Theory, with 
results indicating that the scale yields both valid and reliable results. 
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Questionnaire : 

Description: This self-report requires the teacher to respond to eight questions 
aligned with various INTASC Principles and is designed to elicit specific 
instances of behaviour that reflect the targeted Principles. 

Status: This form has been tested at three universities and results have been 
calibrated as in the belief scale. 

K-12 Focus Group : 

Description: This process allows children to present their views of the 
teacher’s affect, based on specific questions targeted at various Principles. 

Status: This form was field-tested and found to be too complicated. It was 
rewritten to be more closely aligned with the INTASC Principles. Results 
have been analysed judgmentally with accuracy confirmed. 

Situation Reflection Analysis : 

Description: This psychological technique provides for teachers to reflect on 
provocative pictures that yield inappropriate responses for teachers who have 
dispositional deficits. 

Status: This form is being pilot tested in order to identify anchor responses 
for rubric development and future scoring. 

Teacher Observation : 

Description: This behaviour checklist focuses on a variety of INTASC 
Principles with a heavy focus on professionalism, including timeliness, dress, 
and other professional attributes, as well as respect for diversity, effective 
communication, and other INTASC Principles. 

Status: This form is in draft form and has not yet been field tested. 

Discussion 

This work demonstrates the use of evidence that is standards-based to meet 
assessment and accountability requirements with safety and utility. If institutions were to 
develop credible instruments and procedures to measure dispositions, then they would have a 
stronger and more credible voice in state and federal policy making, where legislators and 
state officials are particularly concerned with accountability, standards, and outcomes 
assessment. This model demonstrates an integration of the professional standards for 
teachers, including their values, attitudes, and beliefs about teaching (CCSSO, 1992); the 
nationally recognized standards for sound measurement (AERA, APA, and NCME, 1999), 
and appropriate measurement techniques that have withstood the test of time. If challenged 
in court, this provides a sound backdrop for the use of professional judgment in making 
credible decisions using accepted practice. That is very different from going to court with the 
judgment of a few faculty based on informal and unsystematic observations. 
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Beyond that, data are useful for improving programs and helping them to prepare 
educators who can teach children effectively, since the data can be aggregated in either raw 
and standard scores. When institutions discover that there are areas of the curriculum (e.g., 
parental involvement), of which candidates are not convinced of the necessity or value, 
institutions can redesign their programs to help candidates learn to value such dispositions. 
Without a systematic approach to measuring each disposition in the standards, such 
information can be missed. Since diversity is one of the INTASC Principles, measures of 
candidates valuing of diversity can also help improve their ability to serve these populations. 

At a policy level, institutions and accreditors need to think about the fundamental 
basis for measurement of dispositions. Is it morality and ethics alone, INTASC Principles 
alone, or a combination of both? We advocate for the combination, but with standards taking 
precedence, since morality and ethics are more controversial and can be easily embedded in 
the Standards. This fundamental decision is a difficult one and is, therefore, necessary at 
various policy levels - local, state, and national. We advocate for, and model, a combination 
of qualitative and quantitative evidence to inform decision making about candidates and 
programs. 

Institutions interested in this process may add to, or revamp, their approach to 
measuring dispositions. Such work can also add to the development of new research and new 
policies. Policy makers may begin to focus on standards, helping to solve the imminent legal 
challenges facing those institutions which do not apply standards to their measurement 
process. Additionally, many NCATE reviews find weaknesses with the validity and 
reliability evidence of their institutional reports. 
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